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Abstract. In this paper, we propose three novel and important methods for the 
registration of histological images for 3D reconstruction. First, possible intensity 
variations and nonstandardness in images are corrected by an intensity standard- 
ization process which maps the image scale into a standard scale where the similar 
intensities correspond to similar tissues meaning. Second, 2D histological images 
are mapped into a feature space where continuous variables are used as high con- 
fidence image features for accurate registration. Third, we propose an automatic 
best reference slice selection algorithm that improves reconstruction quality based 
on both image entropy and mean square error of the registration process. We 
demonstrate that the choice of reference slice has a significant impact on registra- 
tion error, standardization, feature space and entropy information. After 2D his- 
tological slices are registered through an affine transformation with respect to an 
automatically chosen reference, the 3D volume is reconstructed by co-registering 
2D slices elastically. 



1. INTRODUCTION 

2D imaging methods, such as optical microscopy, are still preferable to 3D imag- 
ing methods due to their high level of specificity and high resolution properties. 
Histological sections (slices) obtained through 2D imaging methods provide use- 
ful information for the diagnosis or the study of pathology. 3D volume recon- 
struction from these 2D slices is required in order to fully appreciate anatomical 
structures (T). 

Typically a 3D volume is reconstructed by registering (aligning) the 2D sections 
with respect to a chosen reference and stacking successive aligned sections [2]. 
As the acquisition processes of different 2D histological images are performed 
independently slice misalignment and deformation is often unavoidable. Fig- 
ure [l] shows examples of histological slices with non-cohorent distortions, tears, 
hole and missing parts. The deformation varies from section to section and non- 
cohorent distortions may exist in consecutive sections. Choosing an arbitrary slice 
as a reference slice leads to errors in 3D volume reconstruction, hence, the refer- 
ence slice should be chosen properly not to contain distortions in order to achive 
high quality volume reconstruction [ 3 1 . 

Automatic registration of histological slices are necessary because manual regis- 
tration using interactive alignment is non-reproducible and user dependent, there- 
fore, it cannot be used if the number of sections is large |2KU - Among various auto- 
mated registration methods proposed in the literature, rigid and affine registration 
methods can only handle global deformations and the transformation recovered 
from rigid registration has no clinical significance. Since histological slices change 
smoothly from slice to slice and the section distortions induced by the preparation 
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FIGURE 1. Histological images with non-cohorent distortions, 
tears, hole and missing parts 



process are local in nature [41 [3), accurate alignment of these slices can be achived 
by using elastic registration methods (HHHIZHHIED- 

Although intensity differences in consecutive slices are generally assumed to be 
small and are often ignored in most of the existing nonrigid registration methods, 
it is shown that intensity variations for the same tissues can lead to large regis- 
tration errors and the reconstructed 3D volume will not be smooth [lOj EHH 
l2llll|. Figure 12 demonstrates that consecutive slices may have different bright- 
ness/contrast characteristics even for the same tissue regions. Similar to our pre- 
vious study on MRI [10J, to overcome intensity variations, we perform registration 
of histological images in standard intensity scale where similar intensities repre- 
sent similar tissues. 

To ensure that reconstructed 3D volume represents the full anatomy, one way is 
to superimpose the reconstructed volume onto an unsectioned reference volume, 
if it exists [2). However, the problem with this approach is that a reference vol- 
ume is not always available [4]. In the absence of a reference volume, qualitative 
evaluation of reconstructed volume is error prone and quantitative evaluation is 
often needed. The smoothness of the reconstructed surf ace/ volume can be used 
as a quantitative evaluation method in that case [4 J. Therefore, we use smoothness 
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FIGURE 2. Consecutive slices with different contrast map even for 
the same tissue level 

based measurement method, Correlation Alignment Measure (CAM), to quantify 
reconstructed volume. 

A novel 3D reconstruction method for histological images is described in this 
paper. The method tackles three difficult problems in registration of histological 
images. First, in order to capture intensity variations between slices, we stan- 
dardize histological images (Section[2]l. Second, classical motion estimation based 
affine and locally affine registration methods are used to register images in feature 
space and image space respectively (Section [3). Third, we propose an automatic 
best reference slice (BRS) selection algorithm based on iterative assessment of im- 
age entropy and mean squre error (MSE) of the registration process in order to 
improve the quality of the reconstructed volume (Section 2). Quantitative evalu- 
ation of reconstructed volume is given in Section [5] The paper is concluded with 
future research directions and conclusion (Section[6|. 

2. Standardization of Image Intensity Scale 

Image intensity variations are not only influenced by the distribution of light 
sources, but also the content (different tissues) of the images as different tissues 
show different intensity levels. It is important to transform the image scale into 
the standard intensity scale so that for the same body region, intensities will be 
similar l3l ll^[T2| . This is called the intensity standardization. As can be seen from 
Figure |2 some edges are clearly visible in one image, but are not visible in the 
other. This shows the need for intensity standardization. 

Standardization is a non-linear pre-processing technique which maps image in- 
tensity histogram (scale) into a standard intensity histogram through a training 
and a transformation step. In the training step, a set of images of the same body 
region are given as input to "learn" histogram-specific parameters, or the land- 
marks. In the transformation step, any given image is standardized with the esti- 
mated histogram-specific landmarks obtained from the training step. 

It has been proven that image histograms of the same body region are always of 
the same type, and most of the histograms of biomedical images are bimodal [13]. 
Since the histological rat brain images we study also produce bimodal intensity 
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histograms, we extract histogram specific landmarks according to bimodal distri- 
bution as suggested in (121113) . 

In bimodal histograms, one of the histogram specific landmarks is the mode (/i) 
representing the main foreground object in the image, as depicted in Figure [3ja). 
Other histogram specific landmarks denoted by p\ and P2 are extracted accord- 




FlGURE 3. a) Location of the histogram specific landmarks, 
mi=minimum gray value, m2=maximum gray value, b) The in- 
tensity mapping function for the transformation step 

ing to the range of intensity of interest (IOI) by setting minimum and maximum 
percentiles values, namely pc\ and pc2- For any image F"), we will consider his- 
togram specific landmarks as: 

(1) Lj = {pij,P2j,Vj} , 3 = 1, -,M 

where we can increase the number of histogram specific landmarks by setting 
more percentiles values for the main foreground /i (121IT31 . 

In the training step, for each image, the landmarks Lj = {pij , p2j , fij } obtained 
from the histogram are mapped into the standard scale by mapping intensi- 
ties from [pij,p2j] to [si, sa] where s% and S2 are minimum and maximum intensi- 
ties on the standard scale respectively. The formula for mapping x G [pij,P2j] to 
x' G [si, s 2 ] is the following |12|. 

(2) X' = S 1+ X ~ Plj (8 2 -8i) 

P2 3 - Plj 

Figure |3tb) shows the two linear mappings. The first from [pn : Hj\ to [si,fj, s ] and 
the second from [fii,P2i] to [/i s ,S2]. Overall mapping, Ti(x), from [mu,m2i] to 
[s' u , s 2i ] can be summarized as follows: 

!rU, + (x — lli) ( Sl ~^ B )] if mi," < X < Ui 
\fj, s + (x- fj,i) [ p 2 M _% )] Hm<x<m 2l 

where [.] converts any number y£ Re to the closest integer Y such that Y > y or 
< y. Further details and proofs can be found in il3l . 

Figure S] shows slices before and after standardization. The first row shows the 
original data displayed using the default window setting. The second row shows 
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the same slices after standardization displayed using the "standard" window set- 
tings with the parameters defined in Section[5] 




FIGURE 4. Various Original slices before (first row) and after stan- 
dardization (second row). 

3. Registration in Feature Space 

Registration of histological slices requires serial registration procedure which is 
a combination of transformations. Let Aj<_j be the transformation that warps the 
source image i to the target image j. The transformation Aj<—i is computed serially 
as follows: 

Aj^-i=Aj^j-t°Aj- 1 ^-j-2 0...A i+1 ^ i , if i<j 
(4) Aj<-i = Aj^j + i o Aj + i^ j+2 o ... Ai_i<_j, if i > j 

where o represents composition. 

3.1. Locally Affine Nonlinear Transformation. Since consecutive slices are not 
exactly the same, rather slices vary smoothly locally affine globally smooth (LAGS), 
registration algorithm fits well to the problem. LAGS0 registration algorithm uses 
8-affine parameters to fully represents changes between 2D images. Two of these 
affine parameters are needed to capture local brightness and contrast patterns [8] 
and six affine parameters are used to capture local deformations for 2D images. As 
described in our previous study [101, there is no need to use these 2 affine parame- 
ters because the standardization procedure is used to remove intensity variations 
among the same tissue types. Readers are encouraged to read l8l [TOl to under- 
stand theory of LAGS and the modified algorithm which takes into account the 
standardization procedure. 

Choice of the feature space plays a vital role in image registration especially 
if the similarity metric bases on the optimization function independent of spatial 
information (i.e mutual information). Since these kind of registration methods do 
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not take into consideration the spatial information of pixel/voxel intensity dis- 
tribution/variation, the optimization algorithm may get stuck in local maximum 
resulting in misalignment. Defining a feature space capturing variations of gray- 
level characteristics will overcome the drawbacks of intensity based approaches. 
To align the images globally we used a particular feature space which represents 
an image by continuous variables, called edgeness, and describes the intensity 
variance of a predefined region over the image fT4ll3]. 

3.2. Notation and Formulation. We represent an image (section/ slice) by a pair 
F = (F, g) where F is a two-dimensional (2-D) array of pixels and g is intensity 
function defined on F, assigning an integer intensity value for each pixel o e F. A 
particular feature space that allows the representation of an image by continuous 
variables is called the edgeness space [14J, which describes the intensity variance 
of local regions in an image. At image coordinate ro, the edgeness is represented 
by 

(5) F e = \9(n)-g(r )l 

\ri—r \<r f 

where r/ represents a fixed radius. Note that this is not just to determine whether 
a specific voxel/pixel is edge or not [14J. Instead, within a specified radius r/, the 
image feature content is forced to stay beyond a variation level which prevents 
the registration process from getting stuck in local maxima. Edgeness maps for 
various slices are shown in Figure [5j 

4. Automatic Best Reference Slice Selection 

The quality of the reconstructed 3D volume mostly depends on the choice of the 
reference slice. The reference slice is used as a target image and all the remaining 
slices are being considered as source images to be registered onto the target image. 
If the reference slice is distorted or noisy, reconstructed 3D volume will not be 
optimal. Once the reference slice is identified as target image, registration based 
fusion methodology can be applied for reconstruction Q . 

4.1. Metrics. Selecting best reference slice can be based on high confidence image 
features such as MSE, entropy, edge, texture, color, intensity histograms, etc. 

(1) MSE: In the case of distortions, structural discontinuity is not minimum 
even for the consecutive slices. When affine registration is performed for 
global alignment of images, the optimization procedure tries to minimize 
MSE between images but due to distortion, it will not reach low MSE val- 
ues. Furthermore, it is also known that with small SNR values, alignment 
is difficult, leading to high registration errors. Therefore, MSE can be used 
as a tool for checking whether the slices are distorted or not. While high 
MSE values indicate most probably distorted and noisy slices, low MSE 
values indicate strong similarity between consecutive images. 

(2) Edge: In feature space, we emphasise edgeness features of an image by 
mapping image space into the feature space where edgeness parameters 
hold both edge information and spatial variations of pixel intensities over 
all regions in the image. Therefore, we assume that MSE between any 
image pair already includes high confidence information related to edges. 
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FIGURE 5. Histological images and their edgeness maps with ra- 
dius 1 77 1 =3 pixels 

(3) Contrast/Brightness: Contrast/brightness patterns also play important role 
in image contents. Since standardization method has been used to correct 
intensity variations, intensity for the same tissue is the same for all images. 

(4) Entropy: Entropy is another measure often used to characterise the infor- 
mation content of a data source. It has been used as a metric for image 
registration in the form of mutual information. Large mutual information 
between images implies high similarity and vice versa. 

A common way to define entropy E for the jth image F«) is: 

(6) = -J2Po(F^)log(p (F^)) 

oei 

where p D (.) is the probability density function for intensity value o and / is set 
of all possible gray values in F. It is certain that if the amount of information in 
the image is large, the entropy value will be high. From this point, we propose 
the combination of entropy information with MSE as a high confidence feature 
measure to automatically identify BRS for 3D volume reconstruction. 

4.2. Best Reference Slice (BRS). To find the BRS according to the feature sets de- 
fined in Section l4~Tl an iterative framework is developed to compare slices that are 
not consecutive and to avoid errors arising from noisy and distorted images. The 
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framework divides slices into subvolumes. Let M be total number of slices, 14 the 
kth subvolumes where k = 1, .., vol and Nk the number of slices in subvolume k 
such that: Ylti N k = M. 

Since MSE is inversely related to the similarity of images and entropy informa- 
tion is directly related to the content of the image, we define: 

(7) BRSk = argmax ^ log 



ijtjev h I \MSE., 



where MSE^j is feature space based mean square error after registering the image 
i with image j and 

(8) j = argmax { } 

for subvolume Vk . 



5. Experiments and Results 

5.1. Rat Brain Data. We have registered a stack of 350 Nissl-stained slices ac- 
quired by cyro-sectioning coronally an adult mouse brain with a resolution of 
590x520 pixels at a resolution of 15/im and 24-bit color format [15 [. 

5.2. Standardization Parameters. Based on the experiments in [12, 13 1, minimum 
and maximum percentile values are set to pci — and pc2 = 99.8 respectively. 
In the standard scale, s\ and S2 are set to si = 1 and S2 — 4095. Examples of 
intensity mapping for a couple of images, source and target, can be found in IfTOl 
where it can be seen that after standardization, histograms are more similar in 
shape and location. It means that intensities have tissue-specific meaning after 
standardization l3l[10l[12l[T3l. 

5.3. Implementation. Briefly, registration is performed initially for slices in each 
subvolume separately (See Section l4~2)l . Three kinds of registration are performed 
in the reconstruction process: rigid, affine and LAGS. MSEs are calculated accord- 
ing to affine registration in edgeness space and is used to select BRSs for each 
subvolume. Affine registration is performed in a serial manner combining trans- 
formation functions. Then, LAGS registration is performed to capture local de- 
formations in each subvolume with respect to the chosen reference. Once LAGS 
registration processes are completed, subvolumes are registered to each other in a 
rigid manner. 

In order to achieve low computational cost and accelerate the registration pro- 
cess, coarse-to-fine multiresolution framework is used. Registration in finer level 
is performed with the result of the previous level as initial condition. This process 
continues until the finest level is reached. To ensure a more accurate solution, we 
perform standardization after each warping/interpolation. Either small or large, 
intensity changes caused by the interpolation are captured by standardization [10 1 . 

5.4. Evaluations. Quantitative evaluation of the results of the reconstruction pro- 
cess is often difficult. It has been shown in (Tfil that an ideal measure of the qual- 
ity of the reconstruction is the smoothness of the reconstructed surfaces. In that 
work, they propose a new measure based on evaluation of smoothness of the re- 
constructed volume called Correspondence Alignment Measure (CAM). 
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TABLE 1. CAM-mean and standard deviation values for the re- 
constructed 3D 





Rigid Reg. 


Affine Reg. 


LAGS Reg. 


Mean 


55.911 


51.832 


45.461 


Std 


12.223 


9.232 


8.833 



The CAM measure relies on the assumption that if a point is perfectly aligned, it 
lies midway between its corresponding points on neighbors' sections. To compute 
the CAM measure for a given image, first of all, corresponding points for specifed 
control points in the image are identified. The associated confidence values in two 
adjacent images are then calculated. If the confidence is greater than a pre-defined 
threshold t, square root of the summation of the deformation vectors are added 
to the cumulative sum. Finally, the cumulative sum is normalized by the number 
of pixels which have contributed. Note that CAM gives one value for each image, 
therefore, mean or standard deviation of CAM values of serial images are needed 
to compare reconstructions. Reconstucted volume is smooth if the mean or the 
standard deviation of CAM measures are low and vice versa. 

Summary of the changes in mean and standard deviation in CAM values is 
given by Table [1] The values in Table [1] are obtained by considering the worst 
case which uses all the slices instead of just a few slices from the middle of the 
stack as defined in |16| , and r is set to 0. Even for the worst case, CAM val- 
ues indicate that a smooth volume is constructed with the proposed framework. 
While mean values dropped by 7.29% and 18.69%, the standard deviation values 
dropped by 24.46% and 27.73% for affine and locally affine registered stacks re- 
spectively with respect to rigid registered stacks. Figure [6] shows CAM values for 
each section. Registered stacks has lower CAM values which means that smoother 
reconstructed surface /volume is obtained. 



Correlation Alignment Measure 
1 00 1 1 1 1 1 




FIGURE 6. CAM for original and registered slices 
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6. Conclusion 

In this study, we present a novel method for reconstructing 3D rat brain vol- 
umes from 2D histological images. The framework bases on three fundamental 
premises. (1) All histological images must be standardized for accurate registra- 
tion leading to 3D volume reconstruction. (2) For accurate and succesful registra- 
tions in consecutive slices, a reliable feature space must be taken into account. (3) 
For automatic 3D volume reconstruction, the reference slice must be chosen prop- 
erly by avoiding slices with high noise, distortions and other factors. To validate 
the reconstructed volume, the smoothness of the volume is considered. Experi- 
mental results indicate that the reconstructed volume is highly accurate. 
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